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ABSTRACT 

Th^ purpose of this study was to demonstrate the 
feasibility of a .cost-effectiveness analysis approach* Ei^ht 
instructional alternatives within three primary readiny^ programs were 
studied in the Lpuisville Public Schools. Data collected covered 
program description and implementation, student characteristics, 
effectiveness measures, and costs. Program implementation data were 
used to adjust effectiveness measures. Students were grouped using 
characteristic data. By dividing the mean adjusted effectiveness 
measures by the per-Jiupil-program costs, a set of cost-effectiveness 
ratios was calculated that could be uBed to compare programs. The 
study generated information that indicates the possible feasibility 
of the approach. The utilization of three of the four data types 
suggested in determining the cQst-ef fectiyeness of instructional 
programs was successfully implemented. The utilization of a fourth 
type, implementation data, was hot adequately accomplished. 
Therefore, the feasibility of the approach was not completely 
demonstrated. However, the study does suggest a high likelihood of 
feasibility and has pinpoijnted apeas of implementation difficulties. 
Hopefully, the results of Ithis effort, both negative and positive, 
will assist in the development of a cost-effective methodology that 
can be used by the educati|i>n practitioner. (Author) 
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_ABSTRACI 



The purpose of this study was to demonstrate the 
feasibility of a cost-effectiveness analysis ap- 
proach. Eight instructional alternatives within 
three primary reading programs were studied in the 
Louisville (Kentucky) Public Schools. Data collected 
covered program description and implementation, 
student characteristics, effectiveness measures, 
and costs. Program implementation data were used ^ 
to adjust effectiveness measures. Students were ..\ 
grouped using characteristic data. By dividing 
the mean adjusted effectiveness measures by the per 
pupil program costs, af set of cost-effectiveness- 
ratios was calculated whidh could be used ,to compare 
programs. 

The study generated information which indicates the 
possible feasibility of the approach. The utiliza- 
tion'of three of the four data types suggested in 
determining the cdst-ef f ecti veness of instructional 
programs was successfully implemented. The utiliza- 
tion of a fourth type, implementation data, was not 
adequately ^accomplished. Therefore, the feasibility 
of the approach was not completely demonstrated. 
However, the study does suggest a high likelihood of 
feasibility and. has pinpointed areas of implementation 
difficulties. Hopefully the results of this effort, 
both negative and positive, will assist in the develop 
ment of a cost-effective methodology which can be 
utilized by the education practitioner. 
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INTRODUCTION 



Cost-effectiveness analysis is a management tool * 
which can be used to generate information for dexjsion- 
makerSi It combines program effectiveness dati with 
cost analysis information in a format which provides 
the decision-maker with data which will aid in making 
better and more realistic decisions. 

The purpose of this study was to test and demonstrate 
the feasibility of a cost-effectiveness approach. The 
approach was designed to facilitate the utilization 
of cost-effectiveness technology by publ i c school 
practitioners. 

The utilization of programmatic information generated 
by the study is liniited to local efforts, but the 
methodological information has wide application. 
Problejns encountered in the implementation of . the '? 
approach are presented to assist others interested in 
the appl ication of cost-effectiveness analysis, the 
description of the implementation including both 
positive and negative results Js viewed as the real 
worth of the stydy. The feasibility of the approach 
was not completely demonstrated, biut the weak points 
have been identified and efforts.are currently under-^ 
way in the further methodol ogi cal development and 
evaluation of the approach. 

The study was performed in the Louisville /Kentucky) 
Public Schools. Three primary (grades 1-3) reading 
programs were selected for cost-effectiveness analysis. 
One of these programs was being implemented in six 
significant variations: three technical variations . 
and two site variations. 

The following three programs and eight variations 
were analyzed. 

1. Diagnostic-Prescriptive-Indivfdualized 
Instruction --Primary 

a. As operated in non-Teacher Corps schools 
(generally referred to as DPI) \ 

1. ) Student cross-age tutored \f 

2. ) f^araprof essional reading aide 

tutored 

3 . ) Both 1) and 2 ) used as ap- 

propriate (regular DPI program) , 
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b. As taught in Teacher Corps schools 
(generally referred to as TC-DPI) 

1 ) Same as 1 . a . 1 ) 

2) Same as l.a.2) 

3) Same as l.a.3) 

2. Portland-Atkinson-Dolf inger Program (generally 
referred to as PAD or Project Action) 

3. Follow-Through (generally referred to as FT) 

Descriptions of each of these programs are included 
in Appendix B. 
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PROCEDURES 

Introduction ' 

The cost-effectiveness methodology used in this study required the 
collection of four categories of data for each of the alternative 
instructional approaches under study: 

1. Program description and implementation data, 

2. Student characteristics .informatfon, 

3. Program effectiveness measures, and 

4. Costs data. 

Program implementation data were used to adjust effectiveness 
measures. The adjusted effectiveness measures were then used to 
compare alternative instructional^ approaches. By dividing the 
eTfectiveness measures by the program costs, a set af cost- 
effectiveness ratios was calculated. The ratios were then used 
in comparing programs. " ■ , 

Student characteristic data were used as categories when presenting , 
study results in the matrix format designed for utilization 
by decision-makers. ^ 

A detailed descripltion of the cost-effectiveness model from which 
this study was adobted is presented in Appendix A. 

the following paragraphs discuss the implementation of the methodology'. 

Program Descriptions and Implementation 

Descriptive data were collected for each of the instructional 
programs being evaluated. Descriptions (see Appendix B) are • 
based on each program's proposals, staff training materials, and 
curriculum materia-ls'. The general program model is included, as 
well as specifics on the instruction of reading for each program. 
In addition, a brief instructional description and/or interview 
format was written, revised, and agreed to by the administrative 
staff of each program for use in the program implementation 
monitoring. Both DPI and Teacher Corps DPI teachers responded 
to the same instructional description. (Copies of these des- 
criptions and interview forms are included in Appendix C.) 

I'n order to monitor program ^ implementation, each teacher in the 
sample program was interviewed by a staff person related to this 
study. (Teachers were paid for the interviews. In the interview 
the teacher was first asked the open-ended question, "How did 
you teach reading to Johnny Jones during this instructional cycle? 
(The name of an actual sample child from that teacher's room was 
used in the question.) The teacher's response was recorded 
verbatim, and no prompting was used by the interviewer. 
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Then the teacher was asked to read the one-page description of the 
program and state what percentage of the time she followed that 
model.' If the response was les'? than rm the teacher was asked 
in what way she deviated trom the model. It was hoped that 
deviations 'from .the model \vou1d be revealed either in the open- 
ended question or in response to tfte program description. 

At the'r^equest -of the program directors, another measure of 
de§i^ of implementation was added. Each program ha? curriculum 
developers on staff trainer^ who are responsible for working 'with 
the teachers in program implementation. Thes.e people were also 
asked to read the program description and rate each of the 
teachers they supervised on degree of implementation. By 
obtaining both ratings, it was hoped that .some balance in ob- 
jectivity would i)e achieved. 

Feasibility of Collecting Implementation Data 

The interview method of monitoring program implementation 
entailed the usp Of four half-time staff members over a period 
of two months. The -lack of flexibility in the elementary school 
teacher's schedule meant that a maximum of two teachers were 
•seen in a givenlday by one interviewer. (The geographical 
di-stance among the 22 schools in the study also added to problems 
. in .scheduling interviews.) The interview itself only took fifteen 
to twenty minutes. ' , ^ 

When the interviews were completed it became evident that neither 
the open-ended re$ponse nor the percentage response were effective 
measures of progrim implementation. The oDen-ended response pro- 
cluced such' a variety of descriptions, even within programs, that 
it was impossible to compare the answers. In two of the programs, 
PAD and Follow-Through, the teachers expressed their procedures- 
much more clearly and their answers were mOre uniform. However, 
within the DPi program, teacher descriptiojns of instruction ' 
varied greatly. TheSe differences could easily be due to the 
'differences in program structures since PAD and Follow-Through 
have a' more specified, detailed approach to instruction 
while DPI allows teachers -more individual freedom within a 
t^road structural framework. 

The percentage figure given by the teacher after reading the 
program description was also judged ineffective in evaluating 
implementation because it failed to differentiate among teachers. 
Over 75% of the teachers rated themselves as achieving 99-100% 
implementation. Those who gave themselves a lower rating generally 
dropped only to the 95-90% mark. The exception was in the 
Follow-Through program where teachers receive regular feedback 
from the staff trainer as to their level of implementation in- 
exactly the same categories* they were asked about during the inter- 
view, thus providing the Foilow-Through teachers with a concrete 
basis for response. (PAD teachers also received regular feedback, 
but the categories were not the same as ^he ones used in these 



interviews. ) 
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The staff trainer or curriculum specialist's evaluation of 
teacher implementation did^ differentiate among teachers 
wittiin programs. The percentages ranged from 50r to 100% with 
most teachers falling in the 8B-95'^ range. Since the trainer 
evaluations did differentiate -among teachers, it was decided not 
to use the teacher self-evaluations and, instead, bas£ program 
implementation data on trainer ratings. -Because no clear breaks 
in the data occurred, actual, percentage ratings were used on 
each teacher rather than a division into levels of implementation. 

I 

It is now recognized that the interview was not a very feasible 
method 'of determining program implementation. Programs that had 
built-in feed-back to teachers were more reliable and pre^/ise, ^ 
but the problem of teacher reluctance to reveal shortcomings was 
always present. Even trainer ratings wei'e susceptible to personal 
bias and certainly varied according to the goals of the trainer 
where set behavioral objectives were not required by the in- 
structional model . - . ' 

Ratings by independent observers in random classrooms over the 
course of the year would have been a 'preferable method of monitoring 
program implementation. However, this would have involved a . 
total immersion into the goals, training and methods of each 
program., "With several programs to monitor, the time and man- 
power needed would have been outside the.' scope of this study. 

The variance in level of impleme/itation suggested by even our i 
imperfect methods leads the researchers to conclude that the 
disparity between the description of a program and its actual 
operation can be very wi<le. Although level of implementation is 
rarely included in a program's evaluation, the findings of this 
study imply that implementation could make a significant dif- 
ference in program effectiveness. It would be worthwhile to 
spend the time to develop more effeqtive methods of measuring 
implementation. 

Sample 

The sample for this study consisted of 220 second graders, and 213- 
third graders in four instructional programs. In- the DPI program, 
12 schools were randomly selected from the 26 schools in the 
project. Each school was then randomly assigned to one of the . 
three treatment groups; tutor assisted, aide assisted, or regular 
program. In each school ten childVen were randomly picked 
from second grade and ten from third grade. The same procedure 
was followed for the three Teacher Corps schools in^DPI, with 
one school being .assigned to each treatment group. Although DPI 
was designed only for children reading a year or more below grade 
level, the sample was chosen from the total classroom population 
• in the DPI schools^ ^ ^ 
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For PAD, and Follow-Through, no treatment groups existed within the 
program so all schools ifi each project were included in the 
study, i.e., three schools in PAD and four schools in; Follow- 
Through.. The random procedure was also used in selecting t€n 
second and tert third graders for the study from each of these^ 
■ schools. ' 

Student Characteristic Data 

Student characteristic data were included in this .study so that 
more refined decisions could'.be made concerning instructional 
programs. Thus, "the- evaluation results should net only explain 
' which programs are most cost-effective ' ''-o which programs 
are most cost-effective for which tyr 'ents. Toward 

this- goal; student characteristic dat. <t collected in the 
fallowing areas: •" , , ^. 

(1) Socio-economic. 

(a) Free lunch qualification status 

(b) Median census tract income 

»•> 

(2) Pre-school educational attendance, e.g., kindergarten. 
Head Start 

(3) Minority/non-minority ; 
' (4) Male/female 

(5) Self-adjustment 
Feasibility of Collecting Student Characteristic Data 
(1) Socio-economic 

(a) Free -lunch status . Free lunch data were collected 
because this information correlates very highly with Federal in- 
come standards -for poverty families and, therefore, would give 
a measure of socio-economic level. 

No centralized records were kept by the school system on the free 
lunch status of students, so the information had to be obtained 
from individual schools. 

On the whole, the data were easy to collect since local school 
personnel knew which children were on free lunch and which were 
not The only problem came in one or two schools where almost al 
of the-c^iildren qualified for free lunch and, therefore, i«ss 
careful rec1>rds were kept. However, the qualification ot 
children for free lunch was pursued as accurately as possible. 



Although there were students in each cf the four programs who ^ 
-were not on free lunch, the majority of the sample students did 
mTal-ify. Since DPI and Follow-Through are in Title I schools, it 
was expected that more students would be from "poverty" families 
than not, and this was the case. Therefore, non-free lunch . 
students Were under-represented in the sample, making it difficul\t 
to draw conclusions about the effectiveness of programs for this 
type of student. 

(b) Medianj:ensu_s tract income. These data were collected 
by obtaining a'ddresseV of the/sampTe'children, identifying their 
census tract, and then extrapolating a median income fjgure. 
Addresses from the current local school records were accepted a's 

accurate. ' 

' " -- / . '' 

Although income derived from census tract information was readily 
obtained, the results did not prove accurate enough to include 
in the cost-effectiveness study. It was hypothesized that median 
income figures would fall in a relatively narrow range since most.- 
of the sample children wera under Title I classification. However, 
the results indicated that a very wide-range of income was pre- 
sent, even within/ a single program's population. Several census • 
tracts had a median income above poverty level. This was certainly 
in contradiction to the large. number of children in those areas 
recorded as qualifying for free lunch. 

There were no other data available that could provide an accuracy 
check on the census tract income figures, so it was not known 
whether there was an error in the data or in the data collection. 
(Whatever the error, it was decided to exclude the median census 
tract income from the student characteristic data, leaving free 
lunch status as the 'indicator of socio-ecoyomic level. 

( 2) Pre-school Educational Attendance 

The School System h^s a kindergarten or Head Start program 
in all of its elementary schools, and all students are eligible to 
attend However, no centralized records have been kept on which 
children attended these programs or alternative private programs. 
Again, the data, had to be collected from local schools. 

If the sample second and third grade children had attended kinder- 
garten or Head Start in the same school which they were presently 
attending, pre-school experience was recorded in the cumulative 
folders Where/ this was not the case, only sporadic attempts had 
been made tp rkord pre-school attendance. Also, it was not 
always indicated whether a non-response on a record form meant that 
the informatibn was not available or that there was no pre-school 
experience. 

When the pre-school record was incomplete, an attempt was made to 
contact parents to check-' the information. This method was also un- 
satisfactory since a substantial number of the parents did not have 

14 
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telephones. In the end, accurate information on pre-school experi- 
ence could not be obtained for sixty-five children in the sample. 
These children were included with the group of children known to 
be lacking pre-school experience. 

•There were also a few children who had attended private nursery 
'school or a day-care program instead of the public kindergarten 
or Head Start program. These children were designated as 
belonging to the group with pre-school experience since they re- 
ceived some group socialization in an institutional setting, whether 
or not a readir>ess program was also used. No attempt was made to 
"examine effectiveness scores in relation to the specific pre- 
school program which each child attended. 

Presence and absence of pre-school experience was relatively 
even across treatment groups except in Follow-Through where only 
nine out of 80 students lacked pre-school experience. Since 
Follow-Thrdugh was conceived as an extension of the Head Start 
experience, such an overload on pre-schaol experience could be , 
■expected for this program. - 

(3) Minoritv/Non-Minority 

Racial classification data were available both from the 
local school and Central Office records so that collecting the 
data presented no problems. However, the usefulness of the data 
was not consisteat across programs. While the DPI and Teacher 
Corps DPI programs had a racial mixture, there was only one white 
student in the Follow-Through sample and only one black student 
in t;.e PAD sample. This inbalance was unavoidable since the two 
programs happen to be located in racially isolated areas. 

Thus, when student characteristics are looked at in connection 
with cost-effectiveness, it would be impossible to relate the 
advantages of PAD or Follow-Through with regard to the race of 
children taught. However, such a comparison of effectiveness 
with regard to race can be made within DPI and Teacher Corps 
DPI alternatives. 

(4) Male/Female 

Designation of students by sex was readily available from 
local schools and Central Office staff. There was a fairly even 
balance of boys and girls across all programs. 

(5) Self -Adjustment 

Self -adjustment measures on each of the sample students 
were collected using the Child Behavior Rating Scale of Russel 
N. Cassel (1962). The scales were designed to be completed by 
teachers, and they cover personal and school adjustment of the 
children,. Teachers were paid for each form they completed; 
the average teacher had no more than five sample children in his 
or her class. The forms were completed in the two weeks following 
the spring achievement testing of the Sc|iool System. 

11 15 



ERIC 



The teacher-completed rating was used rather than a student- 
completed rating due to the large amount of testirig the children 
had already-received and the problems of testing only a few 
children in the classroom. The rating scales were scored by the 
project researchers and then divided into two levels:^ more 
adjusted and less adjusted. The break was made at the 45th 
percentile, using a T-score conversion scale provided by the 
scale's developer. 

Collection of results using this measure was smooth, and the teachers' 
eomments indicated that the instrument was simple to complete. 
This is one of the few instruments on adjustment or self-concept 
for pritnary age children that can be completed by teachers (Buros. 
*1972) ' Its straight-forwardness and clarity seemed very satis- 
factory for a study of this type where many teachers were involved 
in filling out the instrument, and only a letter of instruction 
was used for guidance. A possible limitation of the CBRS is 
that the teacher's expectations of the child's adjustment may be 
measured rather than the child's actual self -adjustment. 

When the rating scale results were used to divide the sample 
students into two levels of adjustment, students from both groups 
were represented in all programs although less disparity occurred 
in the Follow-Through program where only eighteen of the eighty 
sample students were in the less-adjusted group. 

In general, missing data of various kinds on the children in the 
sample (especially test scores and pre-school attendance infor- 
mation) meant that late substitutions needed to be made, "resulting 
in additional data collection work and eventually in dropping seven 
children from the sample. It would be suggested that this student 
characteristic data be collected earlier in the study. so that ■ 
substitutions could be made in time to collect all needed data, 
and/or that data be collected initially on alternative children ^• 
so that substitutions and data collection would be more efficient. 

Effectiveness Measures 

Standardized reading tests regularly administered by the School 
Svstem to all students were used as the pre- and post-tests tor 
evaluating effectiveness. All second graders received the 
California Achievement Test in the spring of 1973 and the 
California Test of Basic Skills in the spring /of 1974. All third 
graders received the California Test of Basic/Skills in the spring 
of 1973 and 1974. The total reading score seated in grade 
equivalents was used as the effectiveness measure. 

The data for grades 2 and 3 were pooled together. Since grades 2 
and 3 were given different tests, the CAT arid CTBS, respectively, 
this practice might have been expected to distort the results 
because of differing scales of measurement/ The scores were, 
however, converted to grade equivalents for data analysis and 
should, therefore, have had comparable units of measurement in the 
two grades. ;1 g 
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For the DPI program, an additional effectiveness measure collected 
was the number of DPI reading skills passed by each student. 
These data were collected from the reading specialists in each 
school after the last. DPI instructional cycle. There are some 
problems with simply counting numbers of skills passed, since 
some skills are easier to complete and some require a longe'' 
period of study for mastery. Nevertheless, one goal of the DPI 
program was for each child to pass a minimum number of skills in 
each instructional cycle regardless of the entry level of the 
child. 

Due to an oversight, DPI reading skill data were not available 
"for all Teacher Corps DPI schools. 

Attendance Data 

One other measure was regarded as important in relation to 
program delivery-student attendance data. If a student was 
absent a high percentage of instructional days, program imple- 
mentation would obviously be at a lower level than for a child 
who missed only a few days of school. Attendance data were 
collected from office personnel in each school. and were also 
available from the- Central Office. Percentage of attendance 
was based on a possible 152- school days, from the beginning of 
school to the date of the spring testing during the 3rd week 
of April. Students in the sample ranged from 0 days absent to 48 
days absent. 

Effectiveness Results 

To obtain the effectivenes<; portion of the cost-effective ratio 
figure, an adjusted gain score was computed for each of the 
sample students. This adjusted score represented an adjustment 
for treatment delivery, as well as level of entry into program. 

BIOMED program 04R and the following formula w^re used to derive 
an adjusted gain score: 

ek = CTB postk-rg (CTB prek - CTB pre) - r'i (PIk-Pl) 

Where • 

ek = adjusted effectiveness scores per student 
CTB postk = a student post-test score on CTBS 
rg = correlation of CTB post to CTB pre 
CTB prek ^ student pre-test score on CTBS 
CT5 pre = mean student pre-test score 
ri = correlation of CTB post to PI 
PIk = Per^k ^ T^k 

PT = average program^ implementation ratio 
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Per^l^ " % of program implementation by teacher as based on 
trainer rating 

TDl = degree of program delivery found by: ' , 
• # of days belonging (absenteeism for each student) » 
total # of days belonging (152) 



This analysis yielded an adjusted effectiveness score for each 
student. These scores plus all other raw data were keypunched onto 
computer cards. A duplicate deck is available on request. 

A chart represerging the student characteristics on one axis 
and the eight pifgram alternatives on the other axis was con- 
structed. (See Appendix D.) Each of the resulting two hundred 
fifty- two cells represented a unique set of student characteristics 
within a, program. 

The students belonging to each cell were .identified and their . 
adjusted gain (effectiveness) score was placed in the cell, a 
mean adjusted gain score was thu^ computed based on the indivi- 
dual student scores and the cell membership. 

-It should be noted that each cell includes both second and third 
grade students. The adjusted scores relate to grade level so. 
that a score of 2.4 translates into a reading level of second 
grade, fourth month. At the time 'of testing, the average second 
grader reading at national norms would score 2.8 since the test 
is .administered in the eighth month of school. 

Last year the mean reading score for a second grade student in 
the Louisville School System was 2.3 and the mean score for a 
third grade student was 3.2. Therefore, it would be expected 
that the composite mean score for second and third grade 
students would fall somewhere within this range. Since the 
second and third grade scores are combined, it is possible tnat 
a program or treatment could have been more effective in one 
gride than the other, and this would not show up in the mean 
scores Although this information should provide a guideline 
for understanding the effectiveness ratios r^PO''*^^' ^^e im- 
Dortant point of reference is the relationship of effectiveness 
scores among treatment groups. The significant questions are how 
does program A compare with program B and program A with prpgram^ 
C, etc. 

The completed chart gave the mean effectiveness score of each 
of the two hundred fifty-two cells followed by the number of 
students (in parentheses). Since there were onl^f 433 students in 
thHSmple (7 third-grade children could not be included because 
no post-test scores were available) and 252 cells, it was an- 
ticipated that there would be empty cells due to the overload on 
certain student characteristics as already outlined. A total or 
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cells were left empty, wfth the majority of the voids occurring 
in the Teacher Corps DPI, PAD, and Follow-Throucih programs 
which had smaller samples. 

In Fi,gure 1, the mean effectiveness scores according to the 
dimensions of pre-school experience and self-adjustment are pre- 
sented for each program. These two characteristics were combined 
^ince they are factors which a school system might attempt to 
itnprove or change. 

FIGURE 1 

Mean Effectiveness Scores According to 
Pre-School Experience and Self -Adjustment , 

Programs 
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Key: K = Pre-school 

NK = No pre-school 
SAi = More self-adjusted \ 
SAo = Less self adjusted 
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Looking at the observed differences, matching K, SAj dgdiriM. 
NK, SAi, and K, SAo agaitist NK, SA?. it is apparent that pn- 
school experience has a positive effect on the reading achif'Vf™»nf 
of students with the same level of self-adjustment across all 
four treatment groups. Pre-school experience had a very largo 
effect on achievement in the PAD program. It is difficult to nuih*- 
valid comparisons of pre-school effects in the Follow-Through pro- 
gram since so few students lacked pre-school experience. 

On the other hand, the self-adjustment ratings of students seem 
to bear less consistent relationship to program effectiveness. 
In DPI and PAD, more-adjusted students have a slight edge, but in 
Teacher Corps DPI it is actually the less-adjusted students who do 
better. In PAD the more-adjusted students do somewhat better than 
less-adjusted students.' Again, it is difficult to compare 
effectiveness in Follow-Through since the ratio of more-adjusted 
to less-adjusted students is 62 to 18, but indications are that more- 
adjusted students do better. 

Although there is little a school system can do about characteristics 
of race, sex, and socio-economic status of their students, it 
is interesting to note program effectiveness with regard to these 
characteristics. This might be particularly useful when a system 
is concerned about a particular type of student, .say low-income, 
white boys, and wants to find a program to fit their needs. 

Figure 2 presents the relationship tetween sex and program effectives 
ness for each program. 

FIGURE 2 • 
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The DPI program alternatives produced only small differences between 
male and female; however, in Teacher Corps DPI all the alternatives 
worked better for the girls. PAD had exactly the same effectiveness 
for girls and boys, while Follow-Through was more effective for 
girls than for boys. 
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Figur*e 3 presents the relationship between race and effectiveness 
' scores for each program. 

FIGURE 3 

Mean Effectiveness Scores 
According to Race 
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As the report previously pointed out, it is not possible to compare 
PAD and Follow-Through with regard to race since there was only one 
black student in the PAP sample and only one white student in the 
Follow-Through -sample. Teacher Corps DPI alternatives are also not 
comparable for the same reason. However, in DPI, blacks did better 
in the aide and regular groups and whites did better in the tutor 
group. 

Figure 4 presents the final student characteristic comparison, that 
of effectiveness with socio-economic status as based on free lunch 
qualification. 

FIGURE 4 

Mean Effectiveness Scores 
According to Free > Lunch Status 
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For DPI and Teacher Corps DPI, socio-economic status makes little 
or no difference to program effectiveness except in DPI aide, 
where higher income students do much better. Higher income students 
also do better in the PAD program, but only slightly better in 
Fol low-Through. 
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If we discard student characteristic^ altogether, and look only 
at the pfogram variations within DPI and Teacher Corps DPI, 
the following overall results can be observed (Figure 5). 

FIGURE 5 

Total Kean Effectiveness Scores 
In DPI and Teacher Corps DPI 
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Again relying on observed differences, within DPI the alternative 
of using readfng aides. was more effective than either the tutor or 
regular alternatives. The regular program alternative, i.e., the 
use of aides and tutors (or neither); was slightly more effective \ 
than the use of only tutors. 

In Teacher Corps DPI the regular alternative was more effective 
than the use of aides Qr tutor? by themselves. Of these two 
alternatives, the use of tutors was more effective than the use ^ 
of aides. It should be noted that each of the Teacher Corps 
alternatives represents only one school . Thus, treatment effects 
could be tied to a unique situation within a school. 

Figure 6 shows the total mean effectiveness scores for each of the 
four programs. 

FIGURE 6 

Total Mean; Effectiveness Scores by 
Program 
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The Follow-Through alternative is observed to be the most effective 
of any other method in the sample. In fact, Follow^Through's 
3.4 effectiveness score is a full year or more ahead of DPI 
tutor, DPI regular, DPI total, and Teacher Corps aide, and is six 
to seven months ahead of the reading effectiveness scores of 
Teacher Corps DPI total, PAD, and the remaining alternatives. 

The second effectiveness measure which was examined was the number 
of DPI reading skills students in .each treatment group passed. 
This measure, of course, does not involve PAD or Follow-Through. 
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Figure 7 shows the mean number of DPI skills passed by students 
in the sample in DPI and Teacher Corps DPI during the school 
year. 



FIGURE 7 

Mean Number of DPI Skills Passed by. Sample Students in 

1973-1974 
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♦information not collected in this school 

The students in the two Teacher Corps groups where data were 
collected (aide and regular) passed more skills than students In 
any of the DPI alternatives. Within Teacher Corps the regular 
group passed more than the aide group. Within DPI the regular group 
passed the fewest number of skills and the tutor group passed , 
the most. It should be noted agpin that merely counting DPI sk111$ 
has limitations as some skills are easier to master than others. 

It is interesting to. look at the* performance of the DPI treat- 
ment groups on both of the effectiveness measures, number of DPI 
skills passed and adjusted gain 'scores. On both. Teacher Corps 
DPI came out higher overall than DPI. Within DPI the tutor 
alternative had the highest'mean of skills passed, whereas that 
treatment was the lowest of the three alternatives on reading 
effectiveness scores. Of the two groups where data were collected 
in Teacher Corps DPI, the regular group did best on both 
effectiveness feasures. 

Costs 

The budgets for each of the programs being studied were the base 
documents for securHng cost information. Actual expenditures for 
each program were collected after the close of the fiscal year. 
School System accounting records were used in determining program 
costs.' 

Cost data pertaining to expenditures which were assumed to be , 
equivalent for all programs were not collected. Examples of costs 
which iare not included are central office administrative costs, ^ 
utilities, custodial services, facility maintenance, principals 
salaries, general fund school office expenses, and fringe benefits. 
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Thus, costs reported are comparable cost^ and are not the 
true costs of the programs. \ 

Teacher costs vary among programs 'due to the School System's 
sa-lary index which takes into consideration experience and level 
of training in determining salaries. An average teacher salary 
was used in computing program costs. Actual classroom size in 
each program was used to determine per pupil teacher costs which 
also accounts for variations between programs. 

Although teachers were not directly involved in the program 
activities 100% of their time, the cost of teachers was not 
prorated. The rationale for including the total cost is based 
on the System's top priority placed on the instruction of 
reading; reading instruction is expected to be an integral , part 
of the total Instructional program. 

Aide costs differ in DPI and Teacher Corps DPI because DPI aides 
were in the program for the full year while Teacher Corps DPI 
aides were not. 

Figure 8 shows the per pupil costs which were computed for each 
program alternative. 
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Cost Effectivene ss_Re^suUj 

Interim results, i .e. , program implementation, student characteris-^ 
tics and effectiveness measures, were described in the previous 
section. ' • 

Figure 9 provides the cost-effectiveness ratios for each program 
alternative. Rat'os are computed by dividing the mean effective- 
ness measure by program cost. The higher the ratio, the more cost- 
effective the alternative. 

FIGURE 9 - . 

Cost-Effectiveness Ratios 
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The most cost-effective reading approach was the PAD program. The 
least cost-effective was the Teacher Corps DPI program alternative 
which used tutors only. (The reader is referred to the following 
section of this report for a discussion of the utilization of this 
data. ) 
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Cone jus 'ions 

The studv was designed to test and demonstrate the feasibility 
of an approach for determining the cost-effectiveness of in- 
structional programs. The approach was to collect, process, and 
analyze four categories of information: descriptive and imple- 
mentation data, student, characteristics, effectiveness measures, 
and costs. 

Program implementation data were collected by interviewing 
teachers and staff trainers participating in the programs. This 
was not a satisfactory method for obtaining data. If resources 
had been available for collecting data- using an observation/ 
monitoring approach, more useful information may have been 
obtained . 

Student characteristic data relating to socio-economics, race, 
sex, pre-school experience, attendance, and self-adjustment 
were collected. Self -adjustment data were collected by teachers 
completing a Child Behavior Rating Scale; an instrument completed 
by the students may have provided more useful and possibly more 
accurate data, but limited study resources constrained the 
utilization of a student-completed instrument. Some pre- 
school experience records were not complete, and it wa-s not possible 
to determine for 15% of the students in the sample if they had had 
any formal pre-school learning experiences. Socio-economic 
information was collected by determining the free lunch status 
of each student. * 

The student characteristic data did not prove to be as useful as 
hope<3, but did demonstrate the positive effects of pre-school 
educational experiences. 

Effectiveness measyres were standardized achievement test scores 
and the number of skills mastered as measured by criterion- 
referenced tes% in DPI. Standardized test scores were obtained 
from school records. The number of skills mastered was obtained 
from program records. 

Comparable cost data for each program were obtained from the 
official school system accounting records. No attempt was made 
to collect true program costs. (As noted previously, true costs 
would include all program costs. Comparable costs exclude those 
costs whicty would be equivalent across the programs.) 

Cost^iTfectiveness ratios suggest that the Portland-Atkinson- 

Dol finger (PAD) program was the most cost-effective of the approaches 

studied, while Fol low- "hrough appears to be the most effective. 
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The Teacher Corps program and the program which trained and 

provided tutors for the DPI program'^have goals which address 

needs beyond those of. the reading programs being studied. No 

attempt was made to prorate the costs of these\supportive/comp1ementary 

programs. The reading cost-effectiveness shoum not be the only 

data used in analyzing the value and effectivene^ss of these programs. 

There were three major factors which the study did not take 
into consideration in collecting, processing, and analyzing data. 
First, PAD and DPI were ^i^ their first and second years of 
operation, respectively. Both were experiencing some start-up 
problems. Follow-Through was in its fifth^^^ear of operation. 
The maturity of program operation may be a crucial* factor in 
analyzing program effectiveness, i.e., Follow-TKrough may have had 
an advantage/Over the other two programs. The second factor which 
was not included was the scope of program operation. PAD Bnd 
Follow-Through operate in three and four schools, respectively. DPI 
has been implemented in 26 schools. The relatively massive scope 
of DPI may be a complementary factor which effects the early 
successes of the program. The third factor which may have affected ^ 
the standardized test scores for PAD students was the number of 
times standardized achievement measurement instruments were 
administered during the year. The multiple use of standardized 
tests may have helped develop test taking skills in PAD students.' 
However, DPI and Follow-Through students were systematically tested 
using criteria referenced Instruments and this activity may have 
helped develop test taking skills similar to those of the PAD stu- 
dents. Hence, it is difficult to determine the comparative effect 
of this factor. 

These factors and previously suimarized implementation problems 
must be considered in utilizing thefindings of this study in 
the decision-making process. (Study-generated information has been 
used in the decision-making process by School System administrators.) 

The study generated information which indicated the possible 
feasibility of the approach. The utilization of three of the four 
data types suggested in determining the cost-effectiveness of 
instructional programs was successfully implemented. The utiliza- 
tion of the fourth type, implementation data, was not adequately 
accomplished. Therefore, the^feasibility of the approach was not 
completely demonstrated. However, the study does suggest a high 
likelihood of feasibility and has pinpointed areas of implementati9n 
difficulties. Hopefully, the results of this effort, both negativ^ 
and positive^ w^ll assist in the development of a cost-effective 
methodology which can be utilized by the education practitioner. 
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APPENDIX A 



Cost-Effectiveness Analysis: 
Primer and Guidelines 



Roy M. Foibes 



Cosi-of fcLlivcnijss analysis provides a conccpiuji 
ffatnework for/maly/ing the cost and effectiveness of 
cducalfonai progtams When propcily implemenicd, 
C(»sl-cffeilivones^ analysis piovides ihe decision- 
makers with d*vla (ciaicd lo ihc* 

• cost of achieving program ob|eciives; 

• o\<M-all olfecliveness of a program in 
adiicMng Us objectives,, and 

• program effectiveness with subgroups of 
sUidenls. 

^hI^ information is valuable in planning now 
programs atul in deicrnMning if existing programs 
sfiuuld he c\[)andi'(), continued,- modified or deleted 
Ihe purpose of this pitmer and set of guidelines is to 
famrlian/o \hr educator with the concept of cost- 
* effectiveness and pu>\ide him with adequate direction 
so that he may iitin/e this technique. Cost-eftective- 
ness -analysis does not make decisions. This remains 
the responsihiiit\ of the educator. Cost-effectiveness 
anal^Niv \'m[)lv provides the decision-maker with dai.i 
whiLh will aid hini in making better and more realistic 
decisions. 

Cosi-elfoLtAcnesv. should not he contused with 
cost-benetit analvsis, Man\ wniers use these terms 
^Vnonvmousl^ • however, for the' purpo^^cs of this 
article,, a dear disinution is n\ide between these two 
concepts Lffcctiveness is a measure of tlic achtese- 
ment o* program obieciives. Fo* e\ample,. if an 
objeelive of a human relations program is that 35 
pervtMU ot al! students succcssfultv complelii^g the 
, program wiH subsequently volunteer for social ser- 
vice, then the ef fOLliveness of the progran) <^an be 
nK'asurcd by the numbei of stud.'frts volunteering, 
rffectiveness is a measurement of program success in 
achieving stated objectives. 

Cost-benetit is an analysis of the cost and the 
resulting monelarv fienelils ol one or mote progiams 
or program conrponents For example, a cosl-benefil 
analysis ut a vocational educ.ition program would 
allempt to ulefJlily all monetary bene! its resulting 
frc^m the'progjam. Some benehls are easily identified,, 
c.g , pcUerUiai mcieased earnuig power and the 
resulting increase m rncome lax revenue. However, 
other l)enelils becomQ more difficult to measure, e.g., 
possible decrease m welfare expenditures, possible 
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decrease in losses dire, to criminal acts .ind ()enefrls 
associated with lultuial contributions ot lirne and 
resources. Cost-benefit anal)'Sis is n(;!mally mote 
difficult to measure than cost-effectiveness .analysis 
The eflectueness me.isure is usualK rT»oie easily 
identified and obtainable than measurement (;f bene- 
fits. It is more difficult lo dvdne the scope of a 
cost-benefit study. The scope of a cost effectiveness 
study is determined by the staled objectives of the 
program components which are to be aPdly/ed. 

Cost-Effectiveness Elements 

There are four essential elements that sliould be 
considered m a cosi-effectiVeness analysis 

• program descriptions, 

• student characieiisiics, 

• effectiveness measures, and 

• costs. 

Progfum descnpnons A piogran^ description 
should include the following iienis, 

• pi ogram objectives-including anticipated 
outcomes; 

• piogram plan -the implemenlalion and 
opeiattonal activities of the progranr, 

• program histoi y; 

• resouice requirements, 

• resour te availability ;\md 

• external constraints, e.g., community pres- 
sures. 

The statement of the objectives is the most 
important part ot a progiam desciipiion, ihe' objec- 
tives of the piogiam should be stated in me.»su{able 
teipis. Behavioial objectives lend ' to siMiplif> the 
process for determining the effectiveness of the 
piograr^i, but they are not entirely ncc^'^sai y. Expies- > 
sive objectives, if stated m measuralylv urm<i, can also 
be used. 

A description of the (mplemeniatiojn arul operat- 
ing plan of a piogram should include the following 
factors:* 

• list (»l program tasks, i e., activities required 
to miplemeiU and opeiate trie piogiam, and 

• pctsonnel responsibilities, i.e., identify per- 
sonnel responsible foi ;perfoiming program 
aLlivilies 

If the progiam is operational and no major changes rn 
tile operation of the piogram aie pLmned.jhcn the 
implementation of the piogr am. would he considered 
as historical dita lir)wever, if the fjrr/ivam has not 
been completely implemented, ;ihen the im|>lemenla' 
tion ()lan should be presented as current diila. Ihe 
implementation and operating plan are leeded as 
process data. If, as m most cases, the effectiveness^ 
n-jeasure is related to the iirip.UM the piogram has on 
studenXs, e.g., studerU achievernent, ihen data pert/iin- 
ing to program j^rocess become exlremely impoj^iant. 
For example^ by, comparing the plu.-^v for the^imple- 
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•neni.ii'">n .uid nper.U'on of a program with historical 
data rrpri'sofittruj aitiiai implementation and 
opcMltoii i)\ \ [uo^farn, ii ts possible to determine if 
the measured effectiveness of the program result? 
from the proj^ianj designed to achieve specified 
oh)ecl!V">, nr a pifr»;rani whose objectives have been 
altered tfH(/Ugfi the implementation and operating 
process. 

This informat'on is extremely important in the 
analysis of tost effectiveness data. To achieve the 
staled objectives of a piu^ram.. a process must be 
designed and implemented If this process is not 
adequately desij^ned to address each of the programs' 
objectives, and if if)e process is not efficiently 
implemented, then the cost-effectiveness analysis of 
the prngran] should iticlude the consideration of these 
process idiosv nciasies and their effect on the attain- 
ment of program objectives. 

As intimated tn the previous discussion, histori- 
cal data pertaining to the implementation and opera- 
tion of a f rograrv' a/e important in cost-effectiveness 
analvsrs Successes and failures of the process should 
be noted., 

Fbstortcal and current operational data are used 
with cost and effectiveness measures in determining 
the ovei-ali va!u*/ of the program and reaching 
decisions f^ir program improvement,, alterations and 
deletions Prc>gram plans and historical data may be 
verbal!\ de<icrihcd However, these should also exist 
in outline foim,, either as a flow diagram or in 
sentence outt.ne 

fhe next two factr>rs which should be included 
in a pr«>gram dosciipiK)n are resource requirements 
and their availamlitv. Resources needed for the 
implementation a vj operation of the program shouHI 
be listed.. Tiie avadahrhty of resources should be 
discussed. IVugram constraints resulting from supply 
deficiencfcs, s'uito-c^f-the-art development lag and 
geograpfucai and souet.d factors should be included 
in the des<.ri{Minf of [>rogram resource requirements 
and a\a!la[>!lilv . 

No description woultl be complete without a 
discussion of external lonstramis which affect the 
design a'ul operalir^n of the program. For example, 
local labor nruons ofirn influence the si/e and scope 
of vocational cm .utis. Also,, community groups are 
becf^nfing nvr ^ .utivc in tlieir concern for quality 
, education Ali^ 'i gh both nf these factors normally 
have positive efli. ts. Uie role which they play in the 
design and opi f u on c.f eilucalional programs should 
be idi'^ilif led and 'cSu ifu-d. 

Sti/c/rn! < h(jrih fcf fstJcs. The second element 
which shf)uld be considered in a cosi-effectiveness 
anah;S's :s the s^i'^) r)f student characteristics used to 
describe the prow.im's i.jrget population. Student 
character'.siK s c^iJJ listed as one of the items of a 
program descript on. However, its importance in 
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cost-effectiveness analysis dictates that it be listed 
s'.parately. Lxamples of student characteristics data 
are. chrortological age, standardized achievement test 
scores, self-concept,, personalitv, sex, grade, peisonal 
preference, e.g.,. goals, expectancies, etc., and socio- 
economic level, e.g., parents' income and education 
level, housing, etc. 

Student characteristics data should be consid- 
ered when comparing the cost and effectiveness of 
educational programs. fVlaking decisions based on 
cosl»effectiveness data without relating effectiveness 
and costs to student characteristics is not v;ise. 

For programs which serve a small target popula- 
tion it may be desirable to collect characteristics data 
on all students. In other cases,> a sample of student 
characteristics would be sufficient for an analysts. 
There are no hard and fast rules.. The important 
factor is that no cost-effectiveness study should be 
considered complete unless the characteristics of the 
students being served are considered.. 

Student characteristics and program descriptive 
data are often not sufficiently considered in cost- 
effectiveness analysis. The name of the analytical 
technique may be responsible for placing emphasis on 
the cost and effectiveness aspects, but the descrip- 
tions of the programs being evaluated and the 
characteristics of the students are equally important, 
factors. 

Effectiveness measures. If the^objectives of the 
program have been stated in measurable terms, then 
effectiveness is a measure of the level of achievement 
of the objectives. Instructional programs are often 
defined in terms of terminal and intermediate (ena- 
bling) objectives. In these cases, the effectiveness of 
.the program would be measured in terms of the 
terminal objectives. 

. Objectives may also be measured in clusters For 
example, a machine shop student may be measured 
on his over-all performance on the lathe instead of 
the teacher trying to measure separately each lathe 
operation. Therefore,, the effectiveness measure 
would be related to modules of instructional objec- 
tives. 

The general criterion for selecting an effective- 
ness measure should be the reliability and validity of 
the measuring instrument, with the m(jst mipoitant 
aspect being, content validity, i.e.,; the meastnemeni 
of the achievement of the stated program objective. 

Although it IS not recommended, cost-cf festive-- 
ness .Analysis can be performed even though program 
objectives have not been stated in measurable terms. 
For example, two alternative reading programs may 
be compared on the basis of cost and effectiveness 
without having stated program objectives. However, it 
would be necessary to make assumptions about the 
objectives of the program. One assumption wouUl be 
that the programs are designed to improve reading 
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' levels, rtieicforo, a re.idin^ test could be given lo the 
sludctits .irul ttie rOMillini; scores used as an effective- 
nest> me^isuie. 

Plafis fo^ the collection of unanticipated out- 
^come (lata also should he developed Although the 
etfcc liveru'ss c)f the prc^^ram is measured via ohjec-- 
lives, ii IS necessary to include in the analysis data 
describing unanticipated program outcomes which 
relate lo the ovet all effectiveness of the program, 

Tfie selection of an effectiveness measure and an 
understanding of all the explicit and implicit assump- 
tions rel.ilcd to iho selection of the elfecliveness 
measuro ate irnpoftant basic steps in any coy-effec- 
tiveness analysis. 

fhe final element in a cost-effectiveness 
study IS the progiam cost, Ihis element is purposely 
listed last It is tfie easiest of the elements to grasp 
Lonccpttiallv and often receives too much emphasis at 
the expense of the otfier three elements, especially 
the ptogKJm descriptions (objectives) and the student 
charactc r jsI«cs data. 

Aliliough conceptuallv simple, the data collec- 
tion and caie^f^ri/.jiion of costs often become com- 
plex. Ihcre Atv' many ways to classify costs of 
education. il s\sienis; I his article will discuss three 
classification schemes tor tl)e puipose of developing 
the rationale tor the data collection procedures 
suggested f{)r use in cost-effectiveness analysis. 

]\)c first SLhcnie consists of two dichotomous 
classifications mdividii.d -society and measutabic-. 
nonmeasurahlo The expenditures authorized by tJie 
local school board arc an example of a measurable 
societal cost, fhis is the cost of education with which 
la\pa\ers ,!re becoming more concerned. These costs 
include the capital and operating funds needed for 
providing loca' educational opportunities. These 
expenses are referred to as measurable individual 
costs, fhey may include transportation, if bus service 
IS not provided hv the school system, and persona! 
school supi^lics, e.g , paper,, pencils, notebooks, lunch 
mone\,. etc. "I he !(»ss of earnmi; power by students 
attending high NchonI is considered by some writers 
to be an indiVK'ual cost Attempts have been made to 
assign monelaf^v values to this "lost" income, thereby 
making it a nK\5Sur.ible cost However, the economic 
effect on wage* ,ind employment opportunities if all 
high scln>ol stu<ients become part of the job market is 
not reliably predictable, therefore, this "cost" should 
be considered iS iK>rnneasurable. An exan)ple of a 
nf)nmeasurable ^fjcil^tal cost is the financial results of 
the misuse of funds due to poor planning. 

The procCiiures \suggesied for use in cost-effec- 
tiveness analvss are ^^limited to investigating only 
measurable socn^tal costs. 

Measiirahk societ.il costs may be classified as 
direct or mdtret t. Dnecl costs are those expenditures 
incurred in providing edr^cational opportunities, e.g.. 



salaries (iiisti uctional and administrative), sup{)lies,, 
textbooks, buildings (construction and maintenance), 
repairs,, utilities, heat,^ debt service, employee bene- 
fits, etc. These arc* the items which aie listed in the 
school system's budget. Indiiect costs a'e those 
expenses considered lo be relaiec' to the operation of 
the scliool system but which do not appear on budget 
requests. A prime example is the "loss" of tax dollars 
which results when school buildings occupy land 
which would have a high lax assessment. Soriie 
cost-effectiveness techniques suggest th.it indirect 
costs should be estimated and included in the total 
costs of the school system. For cost-henefit analysis,- 
as previously described, this may be relevant. How- 
ever, these indirect costs need not be considered in 
cost-effectiveness analysis. 

The costs suggested for analysis in a cost-effec-- 
tiveness vtudy— direct measurable societal costs -may 
be classified as either capital or operating tosts. For 
purposes of cost-effectiveness analysis, capital costs 
may be defined as those expenditures related to tfie 
planning and implementation of educational pro- 
grams. Included in this cost category are initial 
fjfogram planning, buildmg, building renovati )n,; ac- 
quisition of equipment and non-expendable matenaK, 
special training and orientation programs, administra-: 
tive and/or instructional personnel or services ui 
excess of the planned operational level and otiier costs 
related to the planning and ImplerTientatict phase of 
a piogram.. Similar costs associated with the miple- 
mentation of changes lo existing prograrTis are also 
considered lo be capital costs. 

Opeiating costs include those items associated 
with the operation of a program, e.g., salaries, 
supplies, transportation (if pi;cvided by the school 
system), heat,, utilities, employee benefits, debt ser-- 
vice, custodial services,, etc.. 

Capital costs may be amortized and added lo the 
operational costs to determine a total progr im cost.. 
Again, this process may be useful in cost-benefit 
analysis, but the cost-effecttveness analysis suggested 
in this article does not require this step. Capital cost,, 
with amorti;^a[ion rate data, and operating costs may 
be kept separate. This latter method is recommended. 

One more level of cost categorization needs lo 
be explained,. This categorization is concerned \v«lh 
the assignment of costs lo specific progtariis. "1 hree 
types of cost assignments arc recommended duoct 
assignable, prorated per student and pror.ited per 
space. Instructional salaries, supplres and textbooks 
car) be directly related to specific programs. But the 
costs 'of administering the school system and the 
heating of school buildings must be mathematically 
prorated* to assign costs to specific programs. 

.^Proration guidelines arc available frorri the riulhor upon 
request, 
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A case can be made tor comparing only those 
costs which can be charged directly to specific 
programs. This is supported by the concept of 
relevant cost comparisons when performing cost- 
effectiveness analysis. This concept states simply that- 
there is no need to consider those costs v^hich would 
be equally prorated to the programs or components 
being investigated. For example, assume that a 
decision must be made between two alternative 
methods for achieving the same set of program 
objectives. Each alternative services the same number 
of students and requires the same amount of space 
utilization., Nondirect assignable operation costs are 
determined bv prorating on a per student or per space 
basis. Therefore, in the assumed case, these prorated 
costs would be equal. Since the addition of this 
amount to both sets of costs would not alter their 
ranking, this arithmetic exercise becomes academic. 
The direct assignable cost would provide the cost data 
needed for a decision. 

In summary, the types of cost data recom- 
mended for collection are capital and operating co^^ts 
which are direct measurable societal expenditure. 

The following procedures are recommended for 
use in the collection of cost data.. The accounting 
classification structure presented in this discussion is 
intended to be only an example.. It should not be 
used by systems which have different accounting 
structures unless the system's structure does not 
provide necessary coding flexibility to collect pro- 
gram and subprogiam data. Cost-effectivenes?; proce- 
dures should be altered to fit into the accounting and 
budgeting systems of the schorol.: The accounting and 
budgeting system should not be built around the 
cost-effectiveness procedures. Of course, the opti- 
mum procedure is a planning-programming-budgeting 
system designed for a school system with the ctsst- 
effectiveness analysis function as one of its features., 

A manual prepared by the Division of Research 
and Development, Department of Education, Com- 
monwealth of Massachusetts, describe? accounting 
and budgetary procedures for vocational education 
costs. The codes suggested in this document provide 
the flexibility necessary for identifying program and 
subprogram costs For example, 23M-33-l,IOO 
ide/>ti/ies a teaching salary for the accounting and 
computing subpro'^ram of the vocational day school 
office occupations program. This level of coding is 
necessary only for-dircct assignable costs. Costs which 
are prorated or treated as overhead cost do not 
require this degree of ^.pecificity in the assignment of 
codes. For example, the code assigned to an expendi- 
ture reialed to thr neating of the building is 4120. 
Short codes are assumed to have digits omitted to the 
right of the code which appears. 

If more detad is needed, it is possible to add 
additional code fie'ds to the right of the fields already 
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suggested. For example,« if the instructional objectives 
of the accounting and computing subprogram are 
divided into three modules and it was desirable to 
collect cost data at the module level, then the code 
.Ml 4.33- IJ 00-001 may be used to identify the 
teaching salary expense assigned to the first instruc- 
* tional module. 

It is also theoretically possible, and with an 
automated accounting system feasible, to extend the 
system to provide cost data for the following analysis. 
Students are given several alternative 
methods of achieving an instructional ob- • 
jectivc. The instructional alternatives in- 
clude a regular classroom/instructor learn- 
ing experience, a programmed text, a com- 
puter assisted instructional sequence and a 
learning lab experience. Each of tf:ese 
alternatives requires a different set of re- 
sources and hence each alternative has a 
unique cost element.^ If a cost-effectiveness 
analysis is performed to determine which 
of these instructional alternatives should be 
provided, it would be necessary to have 
cost data broken down to the instructional 
alternative level. 

However, it is not necessary to maintain the 
above level of detail unless the students are provided 
with instructional alternatives. The level of detail 
which is desirable is a function of: 

• the instructional methods used, e.g., tradi- 
tional, individualized instruction, etc., 

• the approach employed in the stating of 
instructional objectives, e.g., behavioral 
stated objectives, expressive objectives, 
clusters (modules) of objectives, etc.;' and 

• the method used for prorating expenses. 
The amNjnt of cost data to be collected for a 

cost-effectiveness analysis is dependent upon the 
following two questions:; (1) Are the cost and 
e'ffectiveness comparisons, to be based on direct 
assignable costs or on direct assignable plus prorated 
costs^ If the former is selected, then it is not 
necessary to collect costs which would be prorated. 
However, this choice implies that either the prorated 
costs are approximately equal for the programs (or 
program parts) being compared or that this difference 
in overhead costs will be accounted for using other 
means during the analysis of the data. If the latter 
method is selected, then it is necessary to collxjcl data 
which will be piorated. It may be determined that the 
number of students to be served by the programs will 
remain constant, but that the space requirements 
vary.. In this case it would be appropriate to collect 
direct assignable cost data and the cost data which are 
prorated per space for a relative cost comparison. (2) 
Arc both capital and operating costs to be compared? 
If '^iQ cost-effectiveness analysis is to be used in 

EDUCATIOhlAL TECHNOLOGY/March, 1974 



reaching decisions pertaining to sclcciions of new 
programs or m.^or changes to existing piogtanis, then 
capital costs must be considered. Howevei. it deci- 
sions are to be made pertaining to npcmtKMial 
programs or parts of programs.; then it may not be 
necessary to consider capita! costs. 

Depending upon tlie answers to the above two 
questions, the amount of data to be collected can 
vary between a^ minimum of t1ie direct as<;ignable 
program (or part of a program) op6raiionjl costs to a 
maximum effort recjuiring the collection ol all capital 
and operating cost of the school svstcm. ! he answers 
to these two questions aie deperulenl upon the plan 
and purpose of the cost-effectivencss analysis. 

Data Analysis 

Although the discussion of djta analysis is 
presented followiog the description ot the data 
collection processes, it <;hould bo pointed out tliat 
activities of the analysis pioiedures occur both 
before, during and after data collection Data analysis 
can be divided into three [?hases planning, monitoi- 
ing and analyzing. The planning activities should 
precede the collertion of data Monitoring of data 
occurs during the collection process and the analyzing 
of the data occurs after data collection is complete. 

Planning. Planning for a cost eftectiveness 
analysis does not differ genciicallv fiom other plan-: 
ning processes. Vhe first step is to dct-rne the goals 
and obiectives of the analysis. The puipose and scope 
must be defined. The goals should describe the 
purpose and the <;cope should be identified hv the 
objectives. The critical questions to be raised in 
defining goals and objcctivcfare 

• What prompted the need for a cost-effec- 
tiveness analysis? 

• Which programs (or parts of programs, e g., 
alternat.ve instructional methods) will be 
analyzed^ 

• Are there any constraints implied or 
explicit - placed on the analysts? 

• How will the resulting data analysis be used 
in reaching decisions? 

The answers to these questions will provide tlie 
data needed for c'esigntng the goaK and objectives of 
the cost-effectiveness study. 

The next step is to define the anticipated 
outcomes of the cost-effectiveness procedure. I his 
does not mean that the results of the analysis will be 
anticipated. "Ou comes" refer to the types of data 
that will be available to the decision maker as a result 
of the analysis. The need for certain ^'outcomes*' 
should be explicit m the statement of the obiectives 
of the analysis. 

The third ^tep in the planning process is to 
identify the data requirements and analytical tech- 
niques which will provide the anticipated outcomes. 



Data roqtiiiemenls for each of the four elements 
vsscMiii.ll m a cosi-cffectiveness analysis should be 
detennined. 

Program descriptions, as previously discussed, 
aie a must. These descriptions would contain the . 
program objectives which, in turn, would lead to the 
ideniittcation of the eflecliveness measure(s). Instru- 
ments for collecting effectiveness data could then be 
identified or designed and tested. 

The requirement for student characteristics data 
IS ver> subieciive. An example of the importance and 
use of the cMta may best dtamati/e the need. Assume 
a cost-effectiveness analysis is being conducted to 
determine if one of several instructional alternatives 
for achieving a set of objectives should be deleted. 
The objectives are well defined and appropriate 
effectiveness measurement instruments have been 
selected. Fhe cost of each alternative method has 
been derived. Alternative "B" has a relatively high 
cost when comjiaied with alternative **A/* The 
effectiveness measure indicates that the m^n student 
achievement rs higher for alternative "A" than for 
alierriative '*B." I he following figure illustrates these 
relative results: 

Alternative 

A B 

Cost 1.0 2.5 

Effectiveness 17 1.0 

[he relative comparison indicates that alternative 
"A" both costs less and has a better rate of 
achieveiTient than altcintitive "B." Without ^ny stu- 
dent characteristics data, the choice may be to delete 
altetn.Uive "B." However, an examination of student 
achievement and student characteristics through an 
aptitude treatment interaction icsearch model (Cron- 
bach, 1067) may indicate that certain types of 
students achieve significantly belter with alternative 
"iV than w'ith alternative "A." This adds valuable 
data for use in the decision process. The decision may 
be to keep both alternatives instead of deleting 
alterchilive "B." 

Hie level (jf cost data required is also a function 
of the purposes of the cost-effectiveness analysis. For 
example, if new educational programs are being 
planned and if student characteristics and effective- 
ness data can be simulated, then it may be the goal of 
the study to perform a cost-effectiveness analysis for 
all subprograms within the designed program. Since 
the program would be new, the capital expenditures 
would be appreciable. Therefore, they should be 
•ncluded in the cost study. All operational costs 
would also be included. Costs assignable directly to 
siibprograms would be identified. Assuming variations 
in the number of students each subprogram would 
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serve and the amount of space required for each 
subprogram, all overhead (nor)direct assignable costs) 
expenses would be prorated. Tins example v^ould 
require a maxinjum cost data collection. 

An example where the capital costs would not 
necessarily have to be collected follows Assume the 
existence of space, trained personnel and equipment 
which could be used for several alternative programs, 
such that the implementation and/or operation of 
any of the alternat-ves would not require any 
additional capital expenditure. It would not be 
necessary to consider capital costs even though the 
programs require the available capital resources. The 
cost-effectiveness analysis ^should consider only the 
relative costs— the operational costs-of the alterna- 
tive programs.. It would not be necessary to collect 
cost data for capital resources— these have been 
assumed to be equal. 

The minimum level of data collection, i.e., 
collecting only direct assignable operating costs, 
could result from the following situation. Assume 
that two alternative instructional methods are being 
compared by their cost-effectiveness. Assume that 
each method serves the same number of students and 
requires the same amount of space.. Also assume that 
no new capital expenditures are required. The only 
costs which would be relevant to the analysis would 
be the direct assignable operational costs. 

During the planning stage of a cost^effectiveness 
analysis it is necessary to identify the data require- 
ments. These requirements vary depending on the 
objectives of the analysis. There are no explicit 
guidelines for determining data requirements. The 
above discussion was intended to introduce some of 
the factors which should be considered.. 

Also included in the third step in planning for a 
cost-effective .ess studv is the selection of analytical 
techniques to be used in the analysis The comparison 
of costs and effectiveness is not complex. An example 
of comparing, two alternatives, ^^as previously pre- 
sented in the discussion peitaming to student charac- 
teristics data requirements. Instead of using a relative 
scale, the actual costs and effectiveness measures 
could be used.. The main disadvantage In using 
'"relative"' costs is that the public and school board 
members are accustomed to discussing per pupil costs 
based on total (xpmditurcs. Costs which are not total 
costs but arc only that portion considered necessary 
*' for a cost-effediveness comparison may prove to be 
confusing. 

, The plans for a cost-effectiveness study should 
also include approaches for achieving staff partici- 
pation in the implementation of the process. The 
successful use of cost-effectiveness techniques is 
strongly dependent upon the quality of the data used 
in the analysis. The data quality is determined by its 
availability and the persons lesponsible for collecting 



the data. The availability of data is a factor which can 
be determined by the person responsible for planning 
the analysis and he can make the necessary planning 
adjustments. 

The collection of student characteristics and 
cost dnta normally requires the involvenient of a few 
individuals, i e., the guidance counselor and finance 
officer.. However, the collection of effectiveness 
measures and program descriptions can involve many 
indiviauals. Hence, the planning; for the active partici- 
pation of these individuals must include approaches 
for developing a strong understanding and appreci- 
ation for the concept of cost-effectiveness analysis. 

Some approaches which could be used in achiev- 
ing the above goal are: 

• inservice training sessions which use practi- 
cal problems as examples of how cost- 
effectiveness analysis can be used; 

• involvement of key staff members In a 
declsion-making process which necessitates 
the use of cost-effectiveness analysis; and 

• demonstration by administrators that cost- 
effectiveness can be used to reach better 
decisions in the planning and operation of 
the school. 

Planning the data analysis portion of a cost- 
effectiveness study should occur prior to data colleo 
tion. 

Monitoring.^ The analysis task which occurs 
simultaneously with data collection is the monitoring 
activity.. Steps should be taken to ensure that the 
most reliable data possible are collected. Any prob- 
lems which occur assumptions made during the 
data collection phase should be explicitly recorded. 

Analyzing, The final phase of the data analysis 
procedures occurs following the ,data collection and 
categorization process.. The analytical technique^ 
planned are executed. The data and analytical results 
are then presented for utilization by the decision- 
makers. 

Cost-Effectiveness Decisions 

Cost-effectiveness analysis does not make de* 
cisions. It provides the decision-maker with data 
which will aid him in making better and more realistic 
decisions. The decision process remains the preroga- 
tive of those persons responsible and accountable for 
the planning and ope'^ation of the educational system.; 

Process Summary 

The described cost -effectiveness approach Is 
summarized by listing process activities. These activi-- 
tics are listed in the normal order that they initially 
Occur. Many of the activities are iterative, i.e., they 
aie refined following the completion of subsequent 
activities; therefore, the following list should not be 
considered as a "once through*' checklist. 
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• Determine goals ni)d objectives of the 
analysts 

• List aniicipaled outcomes 

• nctermine analyticii ttchnhtjcs to be used 

• IdcntKy general dat«t icquir»ments 

• Plan staff participatiun 

• Collect program descriptive data (including 
program objectives) 

• Identify effectiveness measures 

• Identify or design and test effectiveness 
measurement instruments 

• Dcteimine student characteristic data re- 
quiiements 

• Detcimine cost dat.i requirements 

• Identify data sources 

• Plan data collection 

, • ' Data collection and categorization (includ- 
ing monitoring of data quality) / 

• Data analysis j, 

• Data utilization for decision-nr(^king 

This process will provide data pertaining to the 
costs and effectiveness of programii designed to 
achieve stated objectives. But this, ^lone, will not 
provide data for determining the sociafl responsiveness 
of the f»ducational system. Cost-effectiveness analysis 
of instructional programs which are not based on the 
community's social and economic needs will only give 
information on how veil the wrong things are being 
achieved. Cost-effectiveness analysis is a **means" by 
which educators can improve tli/e responsiveness of 
the educational system; it should not be an "end" for 
the educational administrator. □ 

/ 
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Program Descriptions 

The DPI reading program was formulated by the teachers and the 
staff of the iLouisville Public Schools and went into effect in the Fall 
of 1972. The school system made a major commitment to improve the 
level, of reading in its schools and devised a Diagnos tic-Prescriptive- 
Individualized Primary Reading Program under Title I auspices. An 
outline of the basic DPI program, followed by an explanation of the 
instructional variations within the program, is given below. This 
out^ne is taken-4ixectly f rom the DPI handbook as revised in June, 
1973. 

The DPI Reading Program is based upon the assum^^tiqn that a child 
will learn most easily and effectively if his learning strengths and weak- 
nesses provide the basis for his instruction. In order to do this^ the first 
step is the clear definition of the student*s learning strengths and weak- 
nesses, or a diagnosis. Then instruction is prescribed for the pupil, 
based on the diagnostic data. The next step is the individualization of the 
student's instruction giving attention to each student's interests, abilities, 
and achievement level. 

The strategy of the DPI Program involves six major elements: 

1. The concentration of resources will be on the primary 
grades and kindergarten. ' In. these grades, the program 
requires total commitment to a developmental rather 
than a remedial reading program and will involve ap- 
proximately 120 classrooms. 

2. For the reading instruction and language development 
time block, from 8:30 to 11:30 a.m. daily, there must 

be a pupil-adult ratio of 8-1. This ration will be achieved 
through the use of supportive personnel in each classroom. 
The supportive personnel includes pa ra- professional 
reading aides and secondary cross-age tutors (SCAT). 

3. Introduction of diagnosis and prescription of individual 
students in reading is an instructional process. There 
will be three-week instructional cycles which include 
diagnosis, prescription, individualized instruction, 
monitoring and feedback. 
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4. The development of multi-approach program streams 

in each primary classroom will develop the capability for 
individualized instruction. The specific techniques, 
materials, and methodology of the reading instruction 
will be determined by the individual school, 

5. There will be an increase in the amount and type of 
instructional and technical expertise available directly 

• to classroom teachers and principals in the area of 

reading. Specifically, a Reading Specialist and Monitoring 
Technician will be assigned to each ten DPI classroom- 
teachers .for continuous assistance. The Central Office 
Reading Curriculum Specialists will be available continuously. 
Crite rion- referenced instructional objectives for reading^ 
skills on various levels will provide the basis for program 
streams and the three-week instructional cycles. 

6. Quality in-service progi^ams geared to the needs of teachers 
and principals will be developed. 

The overall objective of the DPI program is a reduction of one-half 
of the deficiency between 1971 post-test scores on the Total Reading Score 
of the CTBS and grade expectancy. This was to be achieved by spring, 1974, 
as measured by the Total Reading Score CTBS. ' 

Individual objectives for each child are set forth in the prescription 
for a three-week cycle and involve reaching a designated skill level. These 
cyclical criterion-referenced tests will be utilized in the evaluation of program 
effectiveness. 

All DPI programs follow the model as described above. However, 
alternative methods of individualizing instruction are used which have cost 
differences. The instructional alternatives within the DPI program are 
described below. 

Alternative 1: Regular DPI This title refers to the program 
precisely as described in the attached DPI outline. Regular DPI 
utilizes aides or tutors or both, or neither ,^ to meet the reading needs 
of the individual child. . The type of help given is decided by the DPI 
teacher and reading specialist and the pattern of help may be changed 
to meet the needs of the child. 

Alternative Zt Aide Tutored DPI - The instructional plan remains the 
same with the exception that an aide is assigned to the student to supply 
individual assistance. In this alternative only aides provide individual 
instruction; tutors will not be used. Aides are involved in a training 
institute prior to their involvement in DPI classrooms and receive 
additional training during the school year. (See^description of training 
for aides. ) 
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TRAINING INSTITUTE FOW READING AIDES 
Purpor t* 

The purpose of the D. P. I. Trainiiif; Institute for reading aides is 
to give necessary reading skills to paraprofcssionals so that they can 
effectively offer supplemental services to students on a one-to-one or 
small group basis. ^ 

The reading aides will be expected to know reading skills and^ 
corresponding instructional techniques for their role in the D. P. i: 
Primary Reading Program. * 

Organization 

There will be approximately 130 reading aides participating in the 
institute. The training staff will consist of two professional and two 
paraprofessional positions. 

Duration 



The training will operate in two phases. The first phase will be 
held during the first two weeks of the school year. During this period, 
the reading aides will report to the training institute daily for intensive 
preparatory experiences. The second phase of the training institute 
will begin following the initial two week period and continue throughout 
the school year. Small groups of reading aides will be cycled through 
this phase periodically for continuous training. 

Provision in this on-going training institute will be made for aides 
that are new to the program as replacements. 



Design of the Training Institute 

There are five major reading skill components. They are organized 
into the following units of skill development for which modules have been 
prepared: 

Pre-Reading Skills 
Perception Skills 

Comprehension Skills . 
Word Recognition Skills 
Study Skills 

These components are made up of module clusters of varying lengths. 
Reading aides are to progress through the modules at their own rate of 
speed. 
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Alternative 3; Student Cross-Afi^e Tutored DPI - The instructional 
plan remains the same, but a junior high school tutor is assigned 
to supply individual assistance to the DPI student. In this alternative, 
only tutors will provide individual instruction; aides will not be used. 
Tutors are trained under the F.mergency Secondary Aid Act, a 
federally funded program, and they must meet set standards in 
reading before they can tutor. The DPI program pays the tutors an 
hourly wage for their services. 

Alternative 4, 5, &c 6; Teacher Corps DPI - These alternatives will . 
\ be examined under the three models previously described: regular ' 
\pPI, cross -age tutored DPI, and aide-tutored DPI. The / 
rpstructional model remains the same, but the costs are different 
^ s^Snce the classroom teachers are trained and supervised under the 
Teacher Corps program. 

The overall objectives of Teacher Corps include the following: 

1, Achievement of more relevant educational programs in local ^ 
schools. 

2. Collaboration between the universities and the school district 
in the development of experimental programs to mjeet individual 
pupil needs. \ 

3. Improve achievement of pupils in reading and mathematics. 

4, Foster teacher initiative* 

"5. Improves, pupil attitudes and interpersonal skills. 

6. Create racially balanced faculties. ^ 

7. Expand and strengthen in-service programs, 

8. Develop procedures to achieve equilibrium between curriculum 
continuity and local school autonomy in a decentralized organization. 

9. Improve and increase communication and feedback at all levels. 

10. Improve assessment of students entering teacher education 
programs. 

11. Develop more effective means of preparing new teachers. 

There are also two reading programs in the Louisville Schools which 
utilize other types of instructional methods. Since these programs have 
different budgets and different approaches to reading, they were used as 
comparisons to the various DPI alternatives. 

Follow-Through - The Louisvillci Follow-Through Program is operative 
in four schools and in a total of thirty-one first, second, and third grade 
classrooms. The program follows the University of Kansas model for 
Behavior Analysis Classrooms. In the Kansas model, instructional objectives 
are defined and then the skills of individual children are diagnosed to determine 
the discrepancy between the goals and the present skill of each child. The 
token exchange system is used to reinforce the child's motivation to learn 
and to sulcceed. L — - 
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The University of Kansas provides training in the procedures of 
Behavior Analysis through workshops, in-service courses, and consultation. 
As local personnel become proficient in operating the program, the role 
of the university decreases and ni»w staff are locally trained. 

A Behavior Analysis Follow- Through classroom has the following 
characteristics: 



K Classes are team taught.' There is a lead teacher, a teacher 
assistant, and two parent aides. • 

2. To insure a smooth classroom operation, a peiriod is set 
aside each day for team planning sessions. 

3. Planning, formal instruction, and back-up activities are 
the three parts of a daily classroom schedule. 

4. Classes use curriculum materials that: describe the behavior 
the child will be capable of at the end of the sequence; require 
frequent responding by the child; contain clear criteria for a 
"correct" response; allow for individual rates of progress; 
and provide for periodic testing of achievement gains. 

5. Class schedules alternate instruction with token exchange 
activities. The periods of instruction should gradually increase 
in length during the year. 

6. Discipline is maintained by ignoring inappropriate behavior 
while providing heavy and frequent reinforcement for desirable 
behavior. Behavior which cannot be ignored because it too 
dangerous or disruptive is handled through use of the Time- 
Out procedure. ^ 

Lead teachers and training staff sot the following instructional goals 
in reading ''or Follow-Through children for the 1973-74 school year. 



1. Children will maintain a specified rate of progress through 
the programmed material. Eighty percent of the children 

in ninety percent of the classrooms will be on target through- 
___,_-out iTie 1973-74 school year, 

2. Children will acquire the skills specified in the programmed 
materials. Eighty percent of the children in ninety percent 
of the classrooms will acquire functional reading and math 
skills. 

3. Children will demonstrate ability to use the reading skills 
they have acquired. The median child will perform at or 
above grade level in reading in May, 1974, (as measured by 
the California Test of Basic Skills and the Wide-Range 
Achievement Test). 

e 41 
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The Poraand-Atkinson-Dolfinger Project (P,A,DJ -> The final 
instructional alternative to be evaluated under the cost-effectiveness model 
is known by the P. A. D. acronym. The project name represents the schools 
in which the instructional model is used. The project is funded under a 
grant from the U, Department of Justice - Law Enforcement Assistance 
Administration. 

Project P. A.D. is an attempt to reverse the students' ex- 
periences of failure and transform them into success and achievement utiliz-- 
ing a theoretical model which emphasizes a developmental sequence of be- 
havior characterized by successively more combl^x organization and demon- 
stration of skills. \ 

A second assumption of P.A.D. is\that failure to develop 
basic skills, such as reading, and failure to adap^ to the school environment 
as indicated by poor attendance are critical variables in the development of 
dysfunctional and maladaptive behaviors. Thus, it is hoped that successful 
development of basic skills will help to reverse the emergence of such dys- 
functional behavior, 

A final assumptio.. is that behavior is learned, maintained 
and supported by the consequences it generates. Thus, changing a child's 
behavior involves changes in the consequences of his present behavior and 
shaping new behavior based on the skills a child possesses. Changing 
teacher behavior toward a child shoulr^ then, change the child's behavior. 

The theoretical model is transferred to the school program 
in grades 1-6 through the following intervention plan; 

1. Individualized c rite ria- referenced instruction in read- 
ing and math. 

2. A consistent time and space management system. 

3. Positive motivation system. 

4* A^ata based decision -making system, 
5. A "staff trainer" concept. 
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Appendix C 
DPI MODEL 



To teach reading to (name of child) the DPI teacher follows 
the diagnostic-prescriptive model for individualizing instruction. The 
prescription is written after informal observations have been considered 
and after the diagnostic data has been compiled and analyzed by the reading 
specialist. Together, the DPI teacher and reading specialist write a 
prescription for (name of child) , 

The following general guidelines are followed by the teacher for 
formulating the prescription: 

1. Selection of realistic objectives for the student. 

2. Selection of the best modality for learning and then matching 
materials to that modality. 

3. Selection of materials appropriate to the student*s age and 
instructional and interest levels. 



The written prescription includes: 

1. The activities to be taught in conjunction with the core 
program. This includes new skills. 

2. The skills to be reviewed or emphasized in the classroom 
through individualized assignments. 

3. The skills to be drilled by supplemental personnel. 

The DPI teacher has many ways available to teach and review the 
prescribed skills. However, everyday the student is scheduled to spend 
some time in three kinds of groupings: large group instruction, small 
group instruction, and individual study. The length of time spent in each 
activity is scheduled according to the individual needs of the students. 
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INTERVIEW FORMAT - DPI TEACHERS 



ERIC 



Name / 

School ^' 

Grade / 



/ 

1. How did you tf»ach reading to / during 

this instructional cycle? / 



'li Do you have help m^yo^^r classroom Yes No. 

If yes, what kinds of help do you directly supervise or plan 

for. (Distinguish from help the reading specialist is responsible 

for.) 



a. How do you use this help. 



b. How many times a week and^how long during the day would 
a child get this help. 



3. Please read the following general description of the DPI teacher^s^ 
role in teaching reading. What percentage of the time does your 

• teaching follow this model 

How does what you do differ from this description? 
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INTERVIEW FORMAT - FOLLOW -.THROUGH TEACHERS 



Name 

School 

Grade 

1, How did you teach reading to j during this 

instructional earn period? 



4 

2, When you consider your teaching of reading to the children in your 
class, what percentage of the time: 

Arevthe children on task 

ArCiOn-task contacts made 

Do coVitacts contain praise 

Are kokens paired with praise 

Do cbntVcis with prompts also contain descriptive prais e 
Are no dis^provals given ^ 

How does what you do differ? 
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Program Description 



PROJECT ATKINSON, DOLFINGER, PORTLAND 

The classroom component of f^roject Atkinson, Dolfinger, Portland 
consists of five basic components: an individualized curriculum, a class- 
room model, the use of ^positive reinforcement, the use of P.T. charts, 
and the availability of a staff trainer. 

CURRICULUM 

The project has supplied a number of published materials for use by 
the classroom teachers in reading. These materials are to,help the 
teacher individualize his or her curriculum as much as possible. In 
reading, the basic materials are Sullivan Programmed Reading (Grades 1-3) 
Teacher made and other published materials may be used as long as they 
are referenced and related to the basic materials. 

CLASSROOM MODEL 

The classroom model is simply a way for teachers to work with smaller 
groups of studtMits rather than the entire class. The model refers to the 
use of the activity centers and the wheel. It also involves the use of a 
consistent time schedule for every classroom to aid the teacher in time 
management and iinplementation of the different curriculum pieces. 

J^OSITIVE REINFORCEMENT 

This refers tu setting a positive atmosphere for the classroom through 
positive verbal or non-verbal methods or the setting of specific expectations 
Positive reinforcement is used both for behavior control and academic 
motivation. 

USE OF P. T. 

This refers to taking one minute samples of students reading from word 
lists correlated to the reading programs. The data is then put on a P.T. 
chart. It is used as a diagnostic and record keeping system^ 

THE STAFF TRAINERS 

The staff trainer works with teachers on an ongoing basis during the 
year on implementation of the four basic components of the project* They 
are housed in the school, and assigned to work with particular teachers, 
'primarily in the following ways: supplying of materials; giving support; 
providing classroom interventions; and giving feedback. 
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INTKRVIKW FORMA V - l^AD TKACHKRS 



Name 

School 

Grade 



!• IIow did you teach reading to during the first 

Sullivan morning wheel time? 



IL 1. Please read the Curriculum section of the program description 
and answer this question: During your first reading wheel, what 
% of the time is this statement accurate? % 

2. Please read the Classroom Model section of the program description 
and answer this question: During your first reading wheel, what % 
of the time is this statement accurate? % 

3. Please read the Positive Reinforcement section of the program 
description and answer this question: What % of the time during 
your first reading wheel do you utilize this style of interaction? 

% 

4. F^lease read the P, T. section of the program description and answer 
this question: In your class, what is the average number of P.T. 
times per child per week? times 

5. Please read the Staff Trainer section of the program description. 
Rate whether the staff trainer appropriately meets your needs in 
the following areas on a scale of 1 to 7 with 1 being not appropriate 
and 7 being very appropriate. 

Supply of materials 1 2 3 4 5 6 7 

Giving support 1 2 3 4 5 6 7 

Providing classroom 

" interventions 1 2 3 4 5 6 7 

Giving feedback 1 2 3 4 5 6 7 
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APPENDIX D 



MEAN EFFECTIVENESS SCORES 
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KEY 



FL » Free Lunch 
NFL ■ No free lunch 
¥ ■ Pre-school 

NK * Ho pre-school • 
SAi * More self -adjusted 
SA? * self -adjusted 

F » Female 
n « PUrk 
^ « White 



Notf Nun>bfrs ui parenth«»«»»s indicAto N. 



■^8 
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